Image processing apparatus, image processing method, and computer-readable storage medium

ABSTRACT

A method is provided for displaying physical objects. The method comprises capturing an input image of physical objects, and matching a three-dimensional model to the physical objects. The method further comprises producing a modified partial image by at least one of modifying a portion of the matched three-dimensional model, or modifying a partial image extracted from the input image using the matched three-dimensional model. The method also comprises displaying an output image including the modified partial image superimposed over the input image.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2010-224347 filed in theJapan Patent Office on Oct. 1, 2010, the entire content of which ishereby incorporated by reference.

BACKGROUND Description of the Related Art

The present disclosure relates to an image processing device, program,and method.

In the past, various geographic information services that present to auser geographic information in the real world on a computer screen havebeen put into practical use. For example, through the use of common mapretrieval services opened to the public on the Web, a user can easilyobtain a map of a region including locations corresponding to desiredplace-names, addresses, or names of landmarks, etc. On any mapsdisplayed in map retrieval services, it is also possible to place a markon each facility that complies with the keyword designated by a user.

Furthermore, in recent years, the spectacularly improved computerperformance allows not only a two-dimensional, planar map but alsoinformation on three-dimensional space to be treated with a userterminal. For example, Japanese Patent Application Laid-Open No.2005-227590 proposes displaying a two-dimensional map and athree-dimensional image around a current location in parallel in anoutput image from a map guiding device.

SUMMARY

However, in the method by which a three-dimensional image of an overheadview from a certain viewpoint at a designated location is justdisplayed, the viewpoint of the displayed three-dimensional imagediffers from the viewpoint of a user in the real space. Therefore, whenthe user tries to find the desired building or facility, etc. in thereal space, to which he or she wants to go, a comparison of a view thatthe user sees in the real world around him or her and thethree-dimensional image does not yield a match, and thus the user mayhave difficulty locating the desired building or facility, etc., easily.

Therefore, it is desirable to provide a novel and improved imageinformation processing apparatus, image processing method, andcomputer-readable storage medium that facilitate finding an object, suchas a desired building or facility, in the real space.

Accordingly, there is provided an apparatus for displaying physicalobjects. The apparatus comprises a capturing unit for capturing an inputimage of physical objects. The apparatus further comprises a matchingunit for matching a three-dimensional model to the physical objects. Theapparatus also comprises a processing unit for producing a modifiedpartial image by at least one of modifying a portion of the matchedthree-dimensional model, or modifying a partial image extracted from theinput image using the matched three-dimensional model. The apparatusalso comprises a display generator for generating signals representingan output image, the output image including the modified partial imagesuperimposed over the input image.

In another aspect, there is provided a method for displaying physicalobjects. The method comprises capturing an input image of physicalobjects, and matching a three-dimensional model to the physical objects.The method further comprises producing a modified partial image by atleast one of modifying a portion of the matched three-dimensional model,or modifying a partial image extracted from the input image using thematched three-dimensional model. The method also comprises displaying anoutput image including the modified partial image superimposed over theinput image.

In another aspect, there is provided a tangibly-embodied non-transitorycomputer-readable storage medium storing instructions which, whenexecuted by a processor, cause a computer to perform a method fordisplaying physical objects. The method comprises capturing an inputimage of physical objects, and matching a three-dimensional model to thephysical objects. The method further comprises producing a modifiedpartial image by at least one of modifying a portion of the matchedthree-dimensional model, or modifying a partial image extracted from theinput image using the matched three-dimensional model. The method alsocomprises displaying an output image including the modified partialimage superimposed over the input image.

According to the embodiments described above, there are provided aninformation processing apparatus, information processing method, andcomputer-readable storage medium, finding an object, such as a desiredbuilding or facility in the real space, can be facilitated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of a hardwareconfiguration of an image processing device according to an embodiment;

FIG. 2 is a block diagram illustrating an example of a logicalconfiguration of an image processing device according to an embodiment;

FIG. 3 is an explanatory view illustrating an example of an input image;

FIG. 4 is an explanatory view for explaining an example of athree-dimensional model;

FIG. 5 is an explanatory view illustrating an example of a result of aprocess of matching by a matching unit according to an embodiment;

FIG. 6 is an explanatory view for explaining a process of extracting apartial image by an emphasizing-process unit according to an embodiment;

FIG. 7 is an explanatory view for explaining a first example of aprocess of emphasizing by an emphasizing-process unit according to anembodiment;

FIG. 8 is an explanatory view for explaining a second example of aprocess of emphasizing by an emphasizing-process unit according to anembodiment;

FIG. 9 is an explanatory view for explaining a third example of aprocess of emphasizing by an emphasizing-process unit according to anembodiment; and

FIG. 10 is a flowchart illustrating an example of a flow of imageprocessing according to an embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

In the following, an embodiment of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

It is to be noted that the description is described below in accordancewith the following order.

-   -   1. Configuration example of image processing device        -   1-1. Hardware configuration        -   1-2. Logical configuration    -   2. Examples of processes of emphasizing        -   2-1. First example        -   2-2. Second example        -   2-3. Third example    -   3. Example of flow of image processing    -   4. Conclusion

<1. CONFIGURATION EXAMPLE OF IMAGE PROCESSING DEVICE>

First, an example of a configuration of an image processing deviceaccording to an embodiment will be described with reference to FIGS. 1to 6. Examples of the image processing device described herein include apersonal computer (PC), a smartphone, a personal digital assistant, adigital camera, a car-mounted or handheld navigational device, or ahandheld video game machine.

[1-1. Hardware Configuration]

FIG. 1 is a block diagram illustrating an example of a hardwareconfiguration of an image processing device 100 according to the presentembodiment. Referring to FIG. 1, the image processing device 100includes a central processing unit (CPU) 10, a random access memory(RAM) 12, a read only memory (ROM) 14, a bus 16, a storage device 20, animager 22, a display device 24, an input device 26, a global positioningsystem (GPS) module 30, an electronic compass 32, and an accelerationsensor 34.

The CPU 10 generally controls the operation of the image processingdevice 100 in accordance with programs recorded in ROM 14, storagedevice 20, and/or RAM 12. The RAM 12 temporarily stores a program anddata executed by the CPU 10. The ROM 14 may be a nonvolatile memory forstoring the program for operation of the image processing device 100.The bus 16 interconnects the CPU 10, RAM 12, ROM 14, storage device 20,imager 22, display device 24, input device 26, GPS module 30, electroniccompass 32, and acceleration sensor 34.

The storage device 20 includes a tangibly embodied non-transitorycomputer-readable storage medium, such as a hard disk or semiconductormemory, which may store programs or instructions to be executed by CPU10. The storage device 20 may be non-removable or removable. The datastored in the storage device 20 may be dynamically obtained from anexternal database, when the image processing device 100 performs imageprocessing. Moreover, storage device 20 may be embodied in differentremote devices or servers in a cloud computing configuration.

The imager 22 is configured as a camera module with an image pickupdevice, such as a charge coupled device (CCD) or complementary metaloxide semiconductor (CMOS). The display device 24 displays an image on ascreen, such as a liquid crystal display (LCD), organic light emittingdiode (OLED), or plasma display panel (PDP). The input device 26includes a button, keypad, touch panel, or pointing device, etc. for auser of the image processing device 100 to input information to theimage processing device 100.

The GPS module 30 receives the GPS signal transmitted from the GPSsatellite to calculate an actual latitude, longitude, and altitude, etc.where the image processing device 100 is located. The electronic compass32 detects the actual orientation of the image processing device 100.The acceleration sensor 34 detects acceleration caused by a force actingon the image processing device 100.

[1-2. Logical Configuration]

FIG. 2 is a block diagram illustrating an example of a logicalconfiguration of the image processing device 100 according to thepresent embodiment. Referring to FIG. 2, the image processing device 100includes an input-image obtaining unit 110, a positioning unit 120, astorage unit 130, a matching unit 140, an oscillating-movement sensingunit 150, an application unit 160, a user-interface unit 170, and anemphasizing-process unit 180. As used herein the term “unit” may be asoftware module, a hardware module, or a combination of a softwaremodule and a hardware module. Such hardware and software modules may beembodied in discrete circuitry, an integrated circuit, or asinstructions executed by a processor.

(1) Input-image Obtaining Unit

The input-image obtaining unit 110, i.e. capturing unit, obtains, withthe imager 22, a series of input images in which the view in the realworld around a user is captured. That is, an input image of a physicalobject, or objects, in the real world is captured. Then, the input-imageobtaining unit 110 sequentially outputs the obtained input images to thematching unit 140 and emphasizing-process unit 180.

FIG. 3 is an explanatory view illustrating an example of an input imageobtained by the input-image obtaining unit 110. Referring to FIG. 3, atown, as the real space in which a user who carries the image processingdevice 100 (or a vehicle equipped with the image processing device 100,etc.) is present, appears in the input image. There are multiplebuildings or facilities in the town. Each of such objects that appear inthe input image may be a target for a geographic information search inthe application unit 160 described below, or a target for which the useris guided in navigation using a navigational device.

(2) Positioning Unit

The positioning unit 120 determines the current location of the imageprocessing device 100 with the GPS module 30. The positioning unit 120also determines the actual orientation of the image processing device100 with the electronic compass 32. The positioning unit 120 thenoutputs location information, such as the latitude, longitude, andaltitude, and azimuth information of the image processing device 100obtained as a result of the measurement described above, to the matchingunit 140.

(3) Storage Unit

The storage unit 130 stores a three-dimensional model that describes athree-dimensional shape and a location of an object that is present inthe real world. Examples of the object described by thethree-dimensional model include an object relating to theabove-mentioned application function, such as the geographic informationsearch or navigation provided by the application unit 160. Thethree-dimensional shape and the location of objects, such as arestaurant, an office, a school, a station, a communal facility, and alandmark that are present in a town, are described by thethree-dimensional model. The three-dimensional model may be describedusing a language, such as Geography Markup Language (GML) or KML basedon GML.

FIG. 4 is an explanatory view for explaining an example of thethree-dimensional model. Referring to FIG. 4, a three-dimensional model132 is partially depicted by way of example. In the example shown inFIG. 4, the three-dimensional model 132 includes data on nine objects B1to B9 that are buildings respectively. For each of these objects, thethree-dimensional model 132 defines, for example, an identifier of anobject (hereinafter, “object ID”), a name of a building, andthree-dimensional location of vertices that form the shape of thebuilding, and the predetermined texture. In practice, athree-dimensional model stored by the storage unit 130 may include dataon a larger number of objects (in a more widespread area) than a groupof objects illustrated in FIG. 4. Associated with these objects, thestorage unit 130 may also store additional information, such as a ratingby users for a facility, such as a restaurant or hotel, used by users.

It is to be appreciated that storage unit 130 may be embodied indifferent remote devices or servers in a cloud computing configuration,and that the three-dimensional model may therefore be retrieved fromstorage unit 130 via a network in response to a remote request for athree-dimensional model from image processing device 100, instead ofaccessing a local storage of image processing device 100.

(4) Matching Unit

The matching unit 140 associates an object included in athree-dimensional model around a current location determined by thepositioning unit 120, with an object that appears in an input image bymatching feature points of the object included in the three-dimensionalmodel and feature points in the input image. More specifically, in thepresent embodiment, the matching unit 140 first extracts data on objectsaround the current location of the image processing device 100 from athree-dimensional model stored in the storage unit 130 based on locationand azimuth information input from the positioning unit 120. Thematching unit 140 also sets feature points in an input image accordingto a well-known corner detecting method, such as Harris method orMoravec method. The matching unit 140 matches the feature points of thegroup of objects extracted from the three-dimensional model, and thefeature points set in the input image. Accordingly, the matching unit140 can recognize the correspondence of the object included in thethree-dimensional model to the object that appears in the input image.As a result of matching, the matching unit 140 may also determine thedetailed location and posture of the image processing device 100. Thelocation of the image processing device 100 determined by the process ofmatching described here may have a higher accuracy than the locationaccuracy that the positioning unit 120 can achieve with the GPS module30.

FIG. 5 is an explanatory view illustrating an example of a result of aprocess of matching by the matching unit 140. Referring to FIG. 5, manytriangle-shaped feature points are set in each corner of the objectsthat appear in an input image Im01. Each of these feature points in theinput image Im01 matches each corresponding vertex of the objectsincluded in the three-dimensional model 132 in a specific location andposture of the image processing device 100. For example, a feature pointFP1 matches one of the vertices of an object B2 of the three-dimensionalmodel 132. Thus, the matching unit 140 can recognize at what position inan input image each object included in a three-dimensional modelappears.

(5) Oscillating-movement Sensing Unit

The oscillating-movement sensing unit 150, i.e. sensor unit, detectsthat the image processing device 100 is shaken with the accelerationsensor 34. Upon detecting that the image processing device 100 isshaken, the oscillating-movement sensing unit 150 in turn outputs anoscillating-movement sensing signal to the emphasizing-process unit 180.The oscillating-movement sensing unit 150 may determine that the imageprocessing device 100 is shaken, for example, when acceleration actingon the image processing device 100 exceeds a predetermined threshold.

(6) Application Unit

The application unit 160, i.e. processing unit, is a processing blockthat achieves the above-illustrated application function associated withgeographic information service. The application function that theapplication unit 160 provides includes, for example, geographicinformation search function or navigation function. The application unit160 makes, for example, the user-interface unit 170 display anapplication screen. When a user enters a search keyword via theapplication screen, the application unit 160 searches the storage unit130 for the object that complies with the entered search keyword. Theapplication unit 160 outputs object designating information thatdesignates one or more objects specified as the search result, to theemphasizing-process unit 180. The object designating information may be,for example, information that designates an object selected by the useras a destination of navigation.

(7) User-interface Unit

The user-interface unit 170 provides a user interface to use theabove-described application function, to a user. The user-interface unit170 receives user input information, such as a search keyword forgeographic information search function or a destination of navigation,for example, via the input device 26. The user-interface unit 170 alsodisplays the application screen generated by the application unit 160with the display device 24. Furthermore, the user-interface unit 170displays, for example, an emphasized image generated by theafter-mentioned emphasizing-process unit 180 in the application screen.

(8) Emphasizing-process Unit

The emphasizing-process unit 180 generates an emphasized image thatemphatically displays each of one or more designated objects amongobjects included in a three-dimensional model (hereinafter, “designatedobject”) by processing the partial image in an input image correspondingto each of the above-described designated objects. In the presentembodiment, for example, when object designating information is inputfrom the application unit 160, e.g. an identifier of a physical objectis received, the emphasizing-process unit 180 determines whether andesignated object appears in an input image based on the objectdesignating information. Next, when the designated object appears in theinput image, the emphasizing-process unit 180 recognizes the area in theinput image corresponding to the designated object based on a result ofthe above-described matching by the matching unit 140. Theemphasizing-process unit 180 then processes the partial image in therecognized area such that the designated object corresponding to thereceived identifier is emphasized. The partial image processed andgenerated by the emphasizing-process unit 180 is referred to as anemphasized image or emphasized partial image in this specification. Theemphasizing-process unit 180 generates an output image by superimposingthe emphasized image onto the input image and makes the user-interfaceunit 170 display the generated output image using a display generatorthat generates signals representing the output image.

FIG. 6 is an explanatory view for explaining a process of extracting apartial image by the emphasizing-process unit 180 according to thepresent embodiment. An input image Im01 is shown in the top left-handcorner of FIG. 6. By way of example, it is assumed that objectdesignating information input from the application unit 160 designatesan object B7 that appears in the input image Im01. An area R1 that isrecognized as a result of matching by the matching unit 140 and iscorresponding to the designated object B7, is shown in the topright-hand corner of FIG. 6. The emphasizing-process unit 180 extracts apartial image Im11 of such an area R1 from the input image Im01 (see thebottom left-hand corner of FIG. 6). A moving object that appears in theinput image Im01 may overlap on the partial image Im11 in some cases. Inthe example shown in FIG. 6, a person overlaps in front of thedesignated object B7. In that case, the emphasizing-process unit 180generates a partial image Im12 in which the effect of the moving objectis reduced, for example, by averaging multiple partial images Im11 overmultiple frames.

In the next section, three examples of processes of emphasizing by theemphasizing-process unit 180 will be described in detail to generate anabove-described emphasized image from a partial image extracted in thisway.

<2. EXAMPLES OF PROCESSES OF EMPHASIZING>

FIGS. 7 to 9 are explanatory views for explaining an example of aprocess of emphasizing by the emphasizing-process unit 180 according tothe present embodiment respectively.

[2-1. First Example]

In a first example of the process of emphasizing, it is assumed that theobjects B4 and B7 included in the three-dimensional model 132illustrated in FIG. 4 are designated by object designating information.Referring to FIG. 7, an output image Im21 generated by theemphasizing-process unit 180 in the first example is shown. Bycomparison between the output image Im21 and the input image Im01 inFIG. 3, it will be understood that the size of the designated objects B4and B7 (especially height) is extended in the image. Thus, theemphasizing-process unit 180 may emphasize a designated object bymodifying the size of the partial image corresponding to the designatedobject. Instead (or additionally), the emphasizing-process unit 180 mayemphasize a designated object by modifying the shape, color, orbrightness of the partial image corresponding to the designated object,or by modifying a portion of the three-dimensional model correspondingto the designated object.

The emphasizing-process unit 180 may directly generate an emphasizedimage (shaded portions in the output image Im21 of FIG. 7) thatemphasizes each designated object by changing the size, shape, color, orbrightness of the partial image. Instead, for example, theemphasizing-process unit 180 may remove distortion of a partial image inperspective, obtain a texture of each designated object, and paste theobtained texture on the surface of a polygon of each designated objectto generate an emphasized image. Distortion of a partial image inperspective may be removed by performing affine transformation of thepartial image depending on the location and posture of the imageprocessing device 100 (the imager 22) determined by the matching unit140. The use of the texture obtained in this manner also allows, forexample, a portion of appearance of a designated object hidden by otherobjects in an input image to be visible in an output image (for example,some portions hidden by other objects are visible in the example shownin FIG. 9 described below).

[2-2. Second Example]

Also, in a second example of the process of emphasizing, it is assumedthat the objects B4 and B7 included in the three-dimensional model 132illustrated in FIG. 4 are designated by object designating information.By way of example, it is also assumed that the objects B4 and B7 aredesignated as a result of a restaurant search, and two pieces ofadditional information of “a shop name” and “a rating” are associatedwith each object. For example, referring to FIG. 8, the shop name of thedesignated object B4 is “ABC Cafe,” and the rating is two-star. On theother hand, the shop name of the designated object B7 is “Bar XYZ,” andthe rating is three-star. Additional information is not limited to thisexample, and may be any information, such as an assumed budget andcooking style.

In the second example, the emphasizing-process unit 180, first, similarto the first example, emphasizes the designated objects B4 and B7 bymodifying the size (or shape, color, or brightness) of the partial imagecorresponding to these designated objects B4 and B7. Furthermore, theemphasizing-process unit 180 generates a series of emphasized imagesthat represent animation in which a designated object moves. Thisanimation is, for example, animation in which a designated objectshakes. The emphasizing-process unit 180 may generate such animation inwhich a designated object shakes, for example, when anoscillating-movement sensing signal that indicates that the imageprocessing device 100 is shaken is input from the oscillating-movementsensing unit 150.

Referring to FIG. 8, output images Im22, Im23, and Im24 generated by theemphasizing-process unit 180 in the second example are shown. In theoutput image Im22, the above-described designated objects B4 and B7 areboth emphasized using the method by which the size is extended,regardless of the detection result of oscillating-movement by theoscillating-movement sensing unit 150. On the other hand, the outputimages Im23 and Im24 are displayed when the oscillating-movement of theimage processing device 100 is detected. In the output images Im23 andIm24, only the object B7 that is a restaurant rated highly (three-star)is emphasized using the method that uses animation in which a designatedobject shakes.

Thus, the use of animation in which a designated object moves ensuresthat a desired designated object is distinct from the other objects inthe input image. Therefore, a user can find a desired object very easilyby reference to the output image. Animation in which an object moves isnot limited to the example described above and may be animation in whichan object is, for example, elongated and contracted, or jumping.

Also, as discussed previously, multiple designated objects to bepresented to a user and particularly noteworthy designated objects (forexample, a highly-rated restaurant with many stars, or a restaurant thatsuits the user's budget) of them all, can be displayed in such a waythat the user can distinguish between them easily, by classifying thedesignated objects into two or more groups according to additionalinformation (such as the number of stars, or the user's budget) andemphasizing the designated objects using different methods in a phasedmanner according to the above classification. The method by which user'sshaking the image processing device 100 triggers a designated object tobe shaken also achieves an effective user interface that presentsgeographic information that is more intuitive and easy for users toremember.

[2-3. Third Example]

In a third example of a process of emphasizing, it is assumed that theobjects B4, B7, and a object B10 are designated by object designatinginformation. Similar to the second example, it is also assumed that twopieces of additional information of “a shop name” and “a rating” areassociated with each object. In the third example, however, it isassumed that “a rating” is given as a numerical value and a rank orderaccording to the magnitude of the numerical value is given. For example,the rating of the designated object B4 is 2.9, and the rank order isthird place. The rating of the designated object B7 is 3.8, and the rankorder is second place. The rating of the designated object B10 is 4.2,and the rank order is first place.

In the third example, when the multiple ranked objects are designated inthis manner, the emphasizing-process unit 180 modifies the size of apartial image corresponding to each of the multiple designated objectsaccording to the rank order of each designated object. In the exampleshown in FIG. 9, in an output image Im25, the designated object B10 isdisplayed the tallest, the designated object B7 is displayed the secondtallest, and the designated object B4 is reduced to be the smallest ofall the designated objects.

Thus, the modification of the size of a designated object according toadditional information, such as a rank order, given to the designatedobject allows a user to understand not only the location information onthe designated object but also the additional information intuitivelyand easily.

<3. EXAMPLE OF FLOW OF IMAGE PROCESSING>

FIG. 10 is a flowchart illustrating an example of a flow of imageprocessing by the image processing device 100 according to the presentembodiment. Referring to FIG. 10, first, the input-image obtaining unit110 obtains input images in which the view in the real world around auser is captured (step S102). The positioning unit 120 also determinesthe current location and orientation of the image processing device 100(step S104). Next, the matching unit 140 may retrieve athree-dimensional model corresponding to the current location, andobtain data on objects around the current location of the imageprocessing device 100 from the corresponding three-dimensional modelstored in the storage unit 130 (step S106). Next, the matching unit 140matches the feature points set in the input image with the featurepoints of the group of objects included in the obtainedthree-dimensional model (step S108). Next, when object designatinginformation is input from the application unit 160, theemphasizing-process unit 180 distinguishes one or more designatedobjects designated by the object designating information (step S110).Next, the emphasizing-process unit 180 extracts partial images of areasin the input image corresponding to the designated objects using thethree-dimensional model (step S112). Next, the emphasizing-process unit180 generates the emphasized image that emphasizes the designatedobjects by processing and modifying the extracted partial images (stepS114). Alternatively, a portion of the three-dimensional model may bemodified to generate an emphasized image. The emphasizing-process unit180 generates an output image by superimposing the generated emphasizedimage onto the input image (step S116). The output image generated inthis manner is displayed on the screen of the display device 24 via theuser-interface unit 170.

<4. CONCLUSION>

An embodiment of the present disclosure has been described withreference to FIGS. 1 to 10. According to the present embodiment, anemphasized image that emphasizes a designated object is generated anddisplayed by processing a partial image in an input image correspondingto the designated object in a three-dimensional model designated by theapplication function. Therefore, a designated object is emphaticallydisplayed while natural appearance that appears in an input image fromthe viewpoint of the imager 22 mounted on the image processing device100 that a user uses is kept. Accordingly, it can be easy for a user tofind a designated object, such as a desired building or facility, in thereal space.

For example, the use of a series of emphasized images that representanimation in which a designated object moves also allows a user torecognize the desired object very easily. The modification of adesignated object in an output image also allows a user to understandadditional information, such as a rank order, intuitively and easily.

The method by which user's shaking the image processing device 100triggers animation in which a designated object is shaken in an outputimage to be displayed also associates a motion of a user with anemphasized image, and the method can achieve an intuitive andeasy-to-follow user interface that presents geographic information.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An apparatus comprising: circuitry configured to: acquire an input image and a geographic location of physical objects based on a geographic location and orientation of the apparatus at a time when the input image is acquired; match a three-dimensional model to the physical objects, wherein the three-dimensional model describes shapes and locations of real-world objects that are present at the geographic location in the real world associated with the physical objects in the acquired input image; produce a modified partial image by modifying a partial image extracted from the input image using the matched three-dimensional model so as to emphasize an object of interest; and generate signals representing an output image, the output image including the modified partial image superimposed over the input image, wherein the modified partial image corresponds to the emphasized object of interest, and wherein the three-dimensional model comprises a three-dimensional location of vertices forming the shape of one of the physical objects.
 2. The apparatus of claim 1, wherein the circuitry is further configured to produce the modified partial image by changing at least one of a size, a shape, a color, or a brightness of the partial image.
 3. The apparatus of claim 1, wherein the circuitry is further configured to initiate a storage into a computer-readable medium of the three-dimensional model.
 4. The apparatus of claim 1, wherein the three-dimensional model comprises data defining an identifier of one of the physical objects.
 5. The apparatus of claim 4, wherein the circuitry is further configured to produce a modified partial image of one of the physical objects, the modified partial image corresponding to a received identifier of one of the physical objects.
 6. The apparatus of claim 1, wherein the circuitry is further configured to match feature points of the physical objects in the input image with the three-dimensional model using the three-dimensional location of vertices.
 7. The apparatus of claim 1, wherein the circuitry is further configured to retrieve the three-dimensional model corresponding to the physical objects in the input image, based on the geographic location of the apparatus at the time when the input image is acquired.
 8. The apparatus of claim 1, wherein the circuitry is further configured to detect movement of the apparatus.
 9. The apparatus of claim 8, wherein the circuitry is further configured to produce the modified partial image based on the detected movement.
 10. A method, performed via at least one processor, the method comprising: acquiring an input image of physical objects and a geographic location of physical objects based on a geographic location and orientation of an image processing apparatus at a time when the input image is acquired; matching a three-dimensional model to the physical objects, wherein the three-dimensional model describes shapes and locations of real-world objects that are present at the geographic location in the real world associated with the physical objects in the acquired input image; producing a modified partial image by modifying a partial image extracted from the input image using the matched three-dimensional model so as to emphasize an object of interest; and displaying an output image including the modified partial image superimposed over the input image, wherein the modified partial image corresponds to the emphasized object of interest, and wherein the three-dimensional model comprises a three-dimensional location of vertices forming the shape of one of the physical objects.
 11. A tangibly embodied non-transitory computer-readable storage device storing instructions which, when executed by a processor, cause a computer to perform a method for displaying physical objects, comprising: acquiring an input image and a geographic location of physical objects based on a geographic location and orientation of an image processing apparatus at a time when the input image is acquired; matching a three-dimensional model to the physical objects, wherein the three-dimensional model describes shapes and locations of real-world objects that are present at the geographic location in the real world associated with the physical objects in the acquired input image; producing a modified partial image by modifying a partial image extracted from the input image using the matched three-dimensional model so as to emphasize an object of interest; and displaying an output image including the modified partial image superimposed over the input image, wherein the modified partial image corresponds to the emphasized object of interest, and wherein the three-dimensional model comprises a three-dimensional location of vertices forming the shape of one of the physical objects.
 12. The apparatus of claim 1, wherein a presence of an undesired object within the input image is prevented from appearing within the output image by the superimposing of the modified partial image over the input image.
 13. The apparatus of claim 1, wherein the superimposing of the modified partial image over the input image places a focus upon a designated object located at a corresponding portion of the input image.
 14. The apparatus of claim 13, wherein the focus comprises an animation of the designated object.
 15. The apparatus of claim 1, wherein the circuitry is further configured to initiate a capturing of a capturing image, wherein the input image is based on the captured capturing image. 